Video decoding device and method using inverse quantization

ABSTRACT

In a video decoding device, a quantization step size decoding unit calculates a quantization step size that controls a granularity of the inverse quantization by, based on an image prediction, selectively using a mean value of at least a quantization step size assigned to a leftwardly adjacent neighboring image block already decoded and a quantization step size assigned to a upwardly adjacent neighboring image block already decoded or a quantization step size assigned to an image block decoded immediately before.

This application is a Continuation of U.S. application Ser. No.16/371,161, filed Apr. 1, 2019, which is a Continuation of U.S.application Ser. No. 13/984,951, filed Aug. 12, 2013, which is aNational Stage of International Application No. PCT/JP2012/001592 filedMar. 8, 2012, claiming priority based on Japanese Patent ApplicationNos. 2011-051291, filed Mar. 9, 2011 and 2011-095395, filed Apr. 21,2011, the contents of all of which are incorporated herein by referencein their entirety.

TECHNICAL FIELD

The present invention relates to a video encoding technique, andparticularly to a video encoding technique which makes a prediction withreference to a reconstructed image and performs data compression byquantization.

BACKGROUND ART

A typical video encoding device executes an encoding process thatconforms to a predetermined video coding scheme to generate coded data,i.e. a bitstream. In ISO/IEC 14496-10 Advanced Video Coding (AVC)described in Non Patent Literature (NPL) 1 as a representative exampleof the predetermined video coding scheme, each frame is divided intoblocks of 16×16 pixel size called MBs (Macro Blocks), and each MB isfurther divided into blocks of 4×4 pixel size, setting MB as the minimumunit of encoding. FIG. 23 shows an example of block division in the casewhere the color format of a frame is the YCbCr 4:2:0 format and thespatial resolution is QCIF (Quarter Common Intermediate Format).

Each of the divided image blocks is input sequentially to the videoencoding device and encoded. FIG. 24 is a block diagram showing anexample of the structure of the typical video encoding device forgenerating a bitstream that conforms to AVC. Referring to FIG. 24, thestructure and operation of the typical video encoding device isdescribed below.

The video encoding device shown in FIG. 24 includes a frequencytransformer 101, a quantizer 102, a variable-length encoder 103, aquantization controller 104, an inverse quantizer 105, an inversefrequency transformer 106, a frame memory 107, an intra-frame predictor108, an inter-frame predictor 109, and a prediction selector 110.

An input image to the video encoding device is input to the frequencytransformer 101 as a prediction error image, after a prediction imagesupplied from the intra-frame predictor 108 or the inter-frame predictor109 through the prediction selector 110 is subtracted from the inputimage.

The frequency transformer 101 transforms the input prediction errorimage from a spatial domain to a frequency domain, and outputs theresult as a coefficient image.

The quantizer 102 quantizes the coefficient image supplied from thefrequency transformer 101 using a quantization step size, supplied fromthe quantization controller 104, controlling the granularity ofquantization, and outputs the result as a quantized coefficient image.

The variable-length encoder 103 entropy-encodes the quantizedcoefficient image supplied from the quantizer 102. The variable-lengthencoder 103 also encodes the above quantization step size supplied fromthe quantization controller 104 and an image prediction parametersupplied from the prediction selector 110. These pieces of coded dataare multiplexed and output from the video encoding device as abitstream.

Here, an encoding process for the quantization step size at thevariable-length encoder 103 is described with reference to FIG. 25. Inthe variable-length encoder 103, a quantization step size encoder forencoding the quantization step size includes a quantization step sizebuffer 10311 and an entropy encoder 10312 as shown in FIG. 25.

The quantization step size buffer 10311 holds a quantization step sizeQ(i−1) assigned to the previous image block encoded immediately beforean image block to be encoded.

As shown in the following equation (1), the previous quantization stepsize Q(i−1) supplied from the quantization step size buffer 10311 issubtracted from an input quantization step size Q(i), and the result isinput to the entropy encoder 10312 as a difference quantization stepsize dQ(i).dQ(i)=Q(i)−Q(i−1)  (1)

The entropy encoder 10312 entropy-encodes the input differencequantization step size dQ(i), and outputs the result as codecorresponding to the quantization step size.

The above has described the encoding process for the quantization stepsize.

The quantization controller 104 determines a quantization step size forthe current input image block. In general, the quantization controller104 monitors the output code rate of the variable-length encoder 103 toincrease the quantization step size so as to reduce the output code ratefor the image block concerned, or, conversely, to decrease thequantization step size so as to increase the output code rate for theimage block concerned. The increase or decrease in quantization stepsize enables the video encoding device to encode an input moving imageby a target rate. The determined quantization step size is supplied tothe quantizer 102 and the variable-length encoder 103.

The quantized coefficient image output from the quantizer 102 isinverse-quantized by the inverse quantizer 105 to obtain a coefficientimage to be used for prediction in encoding subsequent image blocks. Thecoefficient image output from the inverse quantizer 105 is set back tothe spatial domain by the inverse frequency transformer 106 to obtain aprediction error image. The prediction image is added to the predictionerror image, and the result is input to the frame memory 107 and theintra-frame predictor 108 as a reconstructed image.

The frame memory 107 stores reconstructed images of encoded image framesinput in the past. The image frames stored in the frame memory 107 arecalled reference frames.

The intra-frame predictor 108 refers to reconstructed images of imageblocks encoded in the past within the image frame being currentlyencoded to generate a prediction image.

The inter-frame predictor 109 refers to reference frames supplied fromthe frame memory 107 to generate a prediction image.

The prediction selector 110 compares the prediction image supplied fromthe intra-frame predictor 108 with the prediction image supplied fromthe inter-frame predictor 109, selects and outputs one prediction imagecloser to the input image. The prediction selector 110 also outputsinformation (called an image prediction parameter) on a predictionmethod used by the intra-frame predictor 108 or the inter-framepredictor 109, and supplies the information to the variable-lengthencoder 103.

According to the processing mentioned above, the typical video encodingdevice compressively encodes the input moving image to generate abitstream.

The output bitstream is transmitted to a video decoding device. Thevideo decoding device executes a decoding process so that the bitstreamwill be decompressed as a moving image. FIG. 26 shows an example of thestructure of a typical video decoding device that decodes the bitstreamoutput from the typical video encoding device to obtain decoded video.Referring to FIG. 26, the structure and operation of the typical videodecoding device is described below.

The video decoding device shown in FIG. 26 includes a variable-lengthdecoder 201, an inverse quantizer 202, an inverse frequency transformer203, a frame memory 204, an intra-frame predictor 205, an inter-framepredictor 206, and a prediction selector 207.

The variable-length decoder 201 variable-length-decodes the inputbitstream to obtain a quantization step size that controls thegranularity of inverse quantization, the quantized coefficient image,and the image prediction parameter. The quantization step size and thequantized coefficient image mentioned above are supplied to the inversequantizer 202. The image prediction parameter is supplied to theprediction selector 207.

The inverse quantizer 202 inverse-quantizes the input quantizedcoefficient image based on the input quantization step size, and outputsthe result as a coefficient image.

The inverse frequency transformer 203 transforms the coefficient image,supplied from the inverse quantizer 202, from the frequency domain tothe spatial domain, and outputs the result as a prediction error image.A prediction image supplied from the prediction selector 207 is added tothe prediction error image to obtain a decoded image. The decoded imageis not only output from the video decoding device as an output image,but also input to the frame memory 204 and the intra-frame predictor205.

The frame memory 204 stores image frames decoded in the past. The imageframes stored in the frame memory 204 are called reference frames.

Based on the image prediction parameter supplied from thevariable-length decoder 201, the intra-frame predictor 205 refers toreconstructed images of image blocks decoded in the past within theimage frame being currently decoded to generate a prediction image.

Based on the image prediction parameter supplied from thevariable-length decoder 201, the inter-frame predictor 206 refers toreference frames supplied from the frame memory 204 to generate aprediction image.

The prediction selector 207 selects either of the prediction imagessupplied from the intra-frame predictor 205 and the inter-framepredictor 206 based on the image prediction parameter supplied from thevariable-length decoder 201.

Here, a decoding process for the quantization step size at thevariable-length decoder 201 is described with reference to FIG. 27. Inthe variable-length decoder 201, a quantization step size decoder fordecoding the quantization step size includes an entropy decoder 20111and a quantization step size buffer 20112 as shown in FIG. 27.

The entropy decoder 20111 entropy-decodes input code, and outputs adifference quantization step size dQ(i).

The quantization step size buffer 20112 holds the previous quantizationstep size Q(i−1).

As shown in the following equation (2), Q(i−1) supplied from thequantization step size buffer 20112 is added to the differencequantization step size dQ(i) generated by the entropy decoder 20111. Theadded value is not only output as a quantization step size Q(i), butalso input to the quantization step size buffer 20112.Q(i)=Q(i−1)+dQ(i)  (2)

The above has described the decoding process for the quantization stepsize.

According to the processing mentioned above, the typical video decodingdevice decodes the bitstream to generate a moving image.

In the meantime, in order to maintain the subjective quality of themoving image to be compressed by the encoding process, the quantizationcontroller 104 in the typical video encoding device is generallyanalyzes either or both of the input image and the prediction errorimage, as well as analyzing the output code rate, to determine aquantization step size according to the human visual sensitivity. Inother words, the quantization controller 104 performsvisual-sensitivity-based adaptive quantization. Specifically, when thehuman visual sensitivity to the current image to be encoded isdetermined to be high, the quantization step size is set small, whilewhen the visual sensitivity is determined to be low, the quantizationstep size is set large. Since such control can assign a larger code rateto a low visual sensitivity region, the subjective quality is improved.

As a visual-sensitivity-based adaptive quantization technique, forexample, adaptive quantization based on the texture complexity of aninput image used in MPEG-2 Test Model 5 (TM5) is known. The texturecomplexity is typically called activity. Patent Literature (PTL) 1proposes an adaptive quantization system using the activity of aprediction image in conjunction with the activity of an input image. PTL2 proposes an adaptive quantization system based on an activity thattakes edge portions into account.

When the visual-sensitivity-based adaptive quantization technique isused, it will cause a problem if the quantization step size is oftenchanged within an image frame. In the typical video encoding device forgenerating a bitstream that confirms to the AVC scheme, a differencefrom a quantization step size for an image block encoded just before animage block to be encoded is entropy-encoded in encoding thequantization step size. Therefore, as the change in quantization stepsize in the encoding sequence direction becomes large, the rate requiredto encode the quantization step size increases. As a result, the coderate assigned to encoding of the coefficient image is relativelyreduced, and hence the image quality is degraded.

Since the encoding sequence direction is independent of the continuityof the visual sensitivity on the screen, the visual-sensitivity-basedadaptive quantization technique inevitably increases the code raterequired to encode the quantization step size. Therefore, even using thevisual-sensitivity-based adaptive quantization technique in the typicalvideo encoding device, the image degradation associated with theincrease in the code rate for the quantization step size may cancel outthe subjective quality improved by the adaptive quantization technique,i.e., there arises a problem that a sufficient improvement in imagequality cannot be achieved.

To address this problem, PTL 3 discloses a technique for adaptivelysetting a range of quantization to zero, i.e. a dead zone according tothe visual sensitivity in the spatial domain and the frequency domaininstead of adaptively setting the quantization step size according tothe visual sensitivity. In the system described in PTL 3, a dead zonefor a transform coefficient determined to be low in terms of the visualsensitivity is more widened than a dead zone for a transform coefficientdetermined to be high in terms of the visual sensitivity. Such controlenables visual-sensitivity-based adaptive quantization without changingthe quantization step size.

CITATION LIST Patent Literatures

PTL 1: Japanese Patent No. 2646921

PTL 2: Japanese Patent No. 4529919

PTL 3: Japanese Patent No. 4613909

Non Patent Literatures

NPL 1: ISO/IEC 14496-10 Advanced Video Coding

NPL 2: “WD1: Working Draft 1 of High-Efficiency Video Coding,” DocumentJCTVC-C403, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-TSG16 WP3 and ISO/IEC JTC1/SC29/WG11 3rd Meeting at Guangzhou, China,October 2010

SUMMARY OF INVENTION Technical Problem

However, when the technique described in PTL 3 is used, quantizationadaptive to the visual sensitivity cannot be performed on transformcoefficients that do not fall within a dead zone. In other words, evenwhen the visual sensitivity is determined to be low, the rate ofcoefficient code for the transform coefficients that do not fall withinthe dead zone cannot be reduced. Further, when the quantization stepsize is enlarged, the transform coefficient values after being subjectedto quantization are concentrated near zero, while when the dead zone iswidened, the transform coefficients that do not fall within the deadzone are not concentrated near zero even after being subjected toquantization. In other words, when the dead zone is widened, theentropy-encoding efficiency is insufficient compared with the case wherethe quantization step size is enlarged. For these reasons, it can besaid that there is a problem in typical encoding technology that theassignment of the code rate to a high visual sensitivity region cannotbe increased sufficiently.

The present invention has been made in view of the above problems, andit is a first object thereof to provide a video encoding device and avideo encoding method capable of changing the quantization step sizefrequently while suppressing an increase in code rate to achievehigh-quality moving image encoding. It is a second object of the presentinvention to provide a video decoding device and a video decoding methodcapable of regenerating a high-quality moving image.

Solution to Problem

A video encoding device according to the present invention for dividinginput image data into blocks of a predetermined size, and applyingquantization to each divided image block to execute a compressiveencoding process, comprises quantization step size encoding means forencoding a quantization step size that controls a granularity of thequantization, wherein the quantization step size encoding means predictsthe quantization step size that controls the granularity of thequantization by using a quantization step size assigned to a neighboringimage block already encoded.

A video decoding device according to the present invention for decodingimage blocks using inverse quantization of input compressed video datato execute a process of generating image data as a set of image blocks,comprises quantization step size decoding means for decoding aquantization step size that controls a granularity of the inversequantization, wherein the quantization step size decoding means predictsthe quantization step size that controls the granularity of the inversequantization by using a quantization step size assigned to a neighboringimage block already decoded.

A video encoding method according to the present invention for dividinginput image data into blocks of a predetermined size, and applyingquantization to each divided image block to execute a compressiveencoding process, comprises predicting a quantization step size thatcontrols a granularity of the quantization by using a quantization stepsize assigned to a neighboring image block already encoded.

A video decoding method according to the present invention for decodingimage blocks using inverse quantization of input compressed video datato execute a process of generating image data as a set of image blocks,comprises predicting a quantization step size that controls agranularity of the inverse quantization by using a quantization stepsize assigned to a neighboring image block already decoded.

Advantageous Effects of Invention

According to the present invention, even when the quantization step sizeis changed frequently within an image frame, the video encoding devicecan suppress an increase in code rate associated therewith. In otherwords, the quantization step size can be encoded by a smaller code rate.This resolves the problem that the subjective quality improved by thevisual-sensitivity-based adaptive quantization is canceled out, that is,high-quality moving image encoding can be achieved. Further, accordingto the present invention, since the video decoding device can decode thequantization step size frequently changed by receiving only a small coderate, a high-quality moving image can be regenerated by the small coderate.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1. It depicts a block diagram showing a quantization step sizeencoder in a video encoding device in a first exemplary embodiment ofthe present invention.

FIG. 2. It depicts an explanatory diagram showing an example of an imageblock to be encoded and neighboring image blocks.

FIG. 3. It depicts a block diagram showing a quantization step sizedecoder in a video decoding device in a second exemplary embodiment ofthe present invention.

FIG. 4. It depicts a block diagram showing a quantization step sizeencoder in a video encoding device in a third exemplary embodiment ofthe present invention.

FIG. 5. It depicts a block diagram showing a quantization step sizedecoder in a video decoding device in a fourth exemplary embodiment ofthe present invention.

FIG. 6. It depicts an explanatory diagram showing prediction directionsof intra-frame prediction.

FIG. 7. It depicts an explanatory diagram showing an example ofinter-frame prediction.

FIG. 8. It depicts an explanatory diagram showing an example ofprediction of a quantization step size using a motion vector ofinter-frame prediction in the video encoding device in the thirdexemplary embodiment of the present invention.

FIG. 9. It depicts a block diagram showing the structure of anothervideo encoding device according to the present invention.

FIG. 10. It depicts a block diagram showing a characteristic componentin another video encoding device according to the present invention.

FIG. 11. It depicts an explanatory diagram of a list showing an exampleof multiplexing of quantization step size prediction parameters.

FIG. 12. It depicts a block diagram showing the structure of anotherdecoding device according to the present invention.

FIG. 13. It depicts a block diagram showing a characteristic componentin another video decoding device according to the present invention.

FIG. 14. It depicts a block diagram showing a quantization step sizeencoder in a seventh exemplary embodiment of the present invention.

FIG. 15. It depicts a block diagram showing a quantization step sizedecoder in a video decoding device in an eighth exemplary embodiment ofthe present invention.

FIG. 16. It depicts a block diagram showing a configuration example ofan information processing system capable of implementing the functionsof a video encoding device and a video decoding device according to thepresent invention.

FIG. 17. It depicts a block diagram showing characteristic components ina video encoding device according to the present invention.

FIG. 18. It depicts a block diagram showing characteristic components inanother video encoding device according to the present invention.

FIG. 19. It depicts a block diagram showing characteristic components ina video decoding device according to the present invention.

FIG. 20. It depicts a block diagram showing characteristic components inanother video decoding device according to the present invention.

FIG. 21. It depicts a flowchart showing characteristic steps in a videoencoding method according to the present invention.

FIG. 22. It depicts a flowchart showing characteristic steps in a videodecoding method according to the present invention.

FIG. 23. It depicts an explanatory diagram showing an example of blockdivision.

FIG. 24. It depicts a block diagram showing an example of the structureof a video encoding device.

FIG. 25. It depicts a block diagram showing a quantization step sizeencoder in a typical video encoding device.

FIG. 26. It depicts a block diagram showing an example of the structureof a video decoding device.

FIG. 27. It depicts a block diagram showing a quantization step sizeencoder in a typical video decoding device.

DESCRIPTION OF EMBODIMENTS

Exemplary embodiments of the present invention are described below withreference to the accompanying drawings.

Exemplary Embodiment 1

Like the video encoding device shown in FIG. 24, a video encoding devicein a first exemplary embodiment of the present invention includes thefrequency transformer 101, the quantizer 102, the variable-lengthencoder 103, the quantization controller 104, the inverse quantizer 105,the inverse frequency transformer 106, the frame memory 107, theintra-frame predictor 108, the inter-frame predictor 109, and theprediction selector 110. However, the structure of a quantization stepsize encoder included in the variable-length encoder 103 is differentfrom the structure shown in FIG. 25.

FIG. 1 is a block diagram showing a quantization step size encoder inthe video encoding device in the first exemplary embodiment of thepresent invention. In comparison with the quantization step size encodershown in FIG. 25, the quantization step size encoder in the exemplaryembodiment is different in including a predicted quantization step sizegenerator 10313 as shown in FIG. 1.

The quantization step size buffer 10311 stores and holds quantizationstep sizes assigned to image blocks encoded in the past.

The predicted quantization step size generator 10313 retrievesquantization step sizes assigned to neighboring image blocks encoded inthe past from the quantization step size buffer to generate a predictedquantization step size.

The predicted quantization step size supplied from the predictedquantization step size generator 10313 is subtracted from the inputquantization step size, and the result is input to the entropy encoder10312 as a difference quantization step size.

The entropy encoder 10312 entropy-encodes the input differencequantization step size and outputs the result as code corresponding tothe quantization step size.

Such a structure can reduce the code rate required to encode thequantization step size, and hence high-quality moving image encoding canbe achieved. The reason is that the absolute amount for the differencequantization step size input to the entropy encoder 10312 can be reducedbecause the predicted quantization step size generator 10313 generatesthe predicted quantization step size using the quantization step sizesof neighboring image blocks independent of the encoding sequence. Thereason why the absolute amount for the difference quantization step sizeinput to the entropy encoder 10312 can be reduced if the predictedquantization step size is generated using the quantization step sizes ofthe neighboring image blocks is because there is generally correlationbetween neighboring pixels in a moving image and hence the degree ofsimilarity of quantization step sizes assigned to neighboring imageblocks having high correlation with each other is high whenvisual-sensitivity-based adaptive quantization is used.

A specific operation of the quantization step size encoder in the videoencoding device in the first exemplary embodiment is described below byusing a specific example.

In this example, it is assumed that the image block size as the unit ofencoding is a fixed size. It is also assumed that three image blocksrespectively adjacent leftwardly, upwardly, and diagonally right upwardwithin the same image frame are used as neighboring image blocks usedfor prediction of the quantization step size.

Suppose that the current image block to be encoded is denoted by X, andthree neighboring image blocks A, B, and C are located respectivelyadjacent leftwardly, upwardly, and diagonally right upward to the imageblock X as shown in FIG. 2. In this case, if the quantization step sizein any block Z is denoted by Q(Z) and the predicted quantization stepsize is denoted by pQ(Z), the predicted quantization step size generator10313 determines the predicted quantization step size pQ(X) by thefollowing equation (3).pQ(X)=Median(Q(A),Q(B),Q(C))  (3)Note that Median(x, y, z) is a function for determining an intermediatevalue from three values of x, y, z.

The entropy encoder 10312 encodes a difference quantization step sizedQ(X) obtained by the following equation (4) using signed Exp-Golomb(Exponential-Golomb) code as one of entropy codes, and outputs theresult as code corresponding to a quantization step size for the imageblock concerned.dQ(X)=Q(X)−pQ(X)  (4)

In this example, the three image blocks adjacent leftwardly, upwardly,and diagonally right upward within the same image frame are used as theneighboring image blocks used for prediction of the quantization stepsize. However, the neighboring image blocks are not limited thereto. Forexample, image blocks adjacent leftwardly, upwardly, and diagonally leftupward may be used to determine the predicted quantization step size bythe following equation (5).pQ(X)=Median(Q(A),Q(B),Q(D))  (5)

The number of image blocks used for prediction may be any number ratherthan three, and a mean value or the like rather than the intermediatevalue may be used as the calculation used for prediction may use. Theimage blocks used for prediction are not necessarily to be adjacent tothe image block to be encoded. The image blocks used for prediction maybe separated by a predetermined distance from the image block to beencoded. Further, the image blocks used for prediction are not limitedto image blocks located in the spatial neighborhood, i.e. within thesame image frame, they may be image blocks within any other image framealready encoded.

Further, in this example, it is assumed that the image block to beencoded and the neighboring image blocks are of the same fixed size.However, the present invention is not limited to the case of the fixedsize, and the block size as the unit of encoding may be a variable size.

Further, in this example, encoding is performed based on the Exp-Golombcode to encode the difference between the quantization step size of theimage block to be encoded and the predicted quantization step size.However, the present invention is not limited to use of the Exp-Golombcode, and encoding may be performed based on any other entropy code. Forexample, encoding based on Huffman code or arithmetic code may beperformed.

The above has described the video encoding device in the first exemplaryembodiment of the present invention.

Exemplary Embodiment 2

Like the video decoding device shown in FIG. 26, a video decoding devicein a second exemplary embodiment of the present invention includes thevariable-length decoder 201, the inverse quantizer 202, the inversefrequency transformer 203, the frame memory 204, the intra-framepredictor 205, the inter-frame predictor 206, and the predictionselector 207. However, the structure of a quantization step size decoderincluded in the variable-length decoder 201 is different from thestructure shown in FIG. 27.

FIG. 3 is a block diagram showing a quantization step size decoder inthe video decoding device in the second exemplary embodiment of thepresent invention. In comparison with the quantization step size decodershown in FIG. 27, the quantization step size decoder in the exemplaryembodiment is different in including a predicted quantization step sizegenerator 20113 as shown in FIG. 3.

The entropy decoder 20111 entropy-decodes input code to output adifference quantization step size.

The quantization step size buffer 20112 stores and holds quantizationstep sizes decoded in the past.

Among quantization step sizes decoded in the past, the predictedquantization step size generator 20113 retrieves quantization step sizescorresponding to neighboring pixel blocks of the current image block tobe decoded from the quantization step size buffer to generate apredicted quantization step size. Specifically, for example, thepredicted quantization step size generator 20113 operates the same wayas the predicted quantization step size generator 10313 in the specificexample of the video encoding device in the first exemplary embodiment.

The predicted quantization step size supplied from the predictedquantization step size generator 20113 is added to a differencequantization step size generated by the entropy decoder 20111, and theresult is not only output as the quantization step size, but also inputto the quantization step size buffer 20112.

Such a structure of the quantization step size decoder enables the videodecoding device to decode the quantization step size by receiving only asmaller code rate. As a result, a high-quality moving image can bedecoded and regenerated. The reason is that the entropy decoder 20111only has to decode the difference quantization step size near zero,because the predicted quantization step size comes close to the actuallyassigned quantization step size when the predicted quantization stepsize generator 20113 generates the predicted quantization step sizeusing quantization step sizes of neighboring image blocks independent ofthe decoding sequence. The reason why the predicted quantization stepsize close to the actually assigned quantization step size can beobtained by generating the predicted quantization step size using thequantization step sizes of the neighboring image blocks is because thereis generally correlation between neighboring pixels in a moving imageand hence the degree of similarity of quantization step sizes assignedto neighboring image blocks having high correlation with each other ishigh when visual-sensitivity-based adaptive quantization is used.

The above has described the video decoding device in the secondexemplary embodiment of the present invention.

Exemplary Embodiment 3

Like the video encoding device in the first exemplary embodiment of thepresent invention, a video encoding device in a third exemplaryembodiment of the present invention includes the frequency transformer101, the quantizer 102, the variable-length encoder 103, thequantization controller 104, the inverse quantizer 105, the inversefrequency transformer 106, the frame memory 107, the intra-framepredictor 108, the inter-frame predictor 109, and the predictionselector 110 as shown in FIG. 24. However, the structure of aquantization step size encoder included in the variable-length encoder103 is different from the structure shown in FIG. 25.

FIG. 4 is a block diagram showing a quantization step size encoder inthe video encoding device in the third exemplary embodiment of thepresent invention. As shown in FIG. 4, the structure of the quantizationstep size encoder in the video encoding device in the third exemplaryembodiment of the present invention is the same as the structure of thequantization step size encoder shown in FIG. 1. However, the thirdexemplary embodiment differs from the first exemplary embodiment in thatthe parameter used for image prediction is supplied from the predictionselector 110 shown in FIG. 24 to the predicted quantization step sizegenerator 10313 in the third exemplary embodiment, and in the operationof the predicted quantization step size generator 10313.

Since the operation of the quantization step size buffer 10311 and theentropy encoder 10312 is the same as that of the quantization step sizeencoder in the video encoding device in the first exemplary embodiment,redundant description is omitted here.

The predicted quantization step size generator 10313 uses the imageprediction parameter to select an image block to be used for predictionof the quantization step size from among image blocks encoded in thepast. The predicted quantization step size generator 10313 generates apredicted quantization step size from a quantization step sizecorresponding to the image block selected.

Such a structure enables the video encoding device to further reduce thecode rate required to encode the quantization step size compared withthe video encoding device in the first exemplary embodiment. As aresult, high-quality moving image encoding can be achieved. The reasonis that the quantization step size can be predicted from neighboringimage blocks having high correlation with the image block concernedbecause the predicted quantization step size generator 10313 predictsthe quantization step size using the image prediction parameter.

Exemplary Embodiment 4

Like the video decoding device in the second exemplary embodiment of thepresent invention, a video decoding device in a fourth exemplaryembodiment of the present invention includes the variable-length decoder201, the inverse quantizer 202, the inverse frequency transformer 203,the frame memory 204, the intra-frame predictor 205, the inter-framepredictor 206, and the prediction selector 207 as shown in FIG. 26.However, the structure of a quantization step size decoder included inthe variable-length decoder 201 is different from the structure shown inFIG. 27.

FIG. 5 is a block diagram showing a quantization step size decoder inthe video decoding device in the fourth exemplary embodiment of thepresent invention. As shown in FIG. 5, the structure of the quantizationstep size decoder in the video decoding device in the fourth exemplaryembodiment of the present invention is the same as the structure of thequantization step size decoder shown in FIG. 3. However, the fourthexemplary embodiment differs from the second exemplary embodiment inthat the parameter used for image prediction is supplied from theprediction selector 207 shown in FIG. 26 to the predicted quantizationstep size generator 20313, and in the operation of the predictedquantization step size generator 20113.

Since the operation of the entropy decoder 20111 and the quantizationstep size buffer 20112 is the same as that of the quantization step sizedecoder in the video decoding device in the second exemplary embodiment,redundant description is omitted here.

The predicted quantization step size generator 20113 uses the imageprediction parameter to select an image block to be used for predictionof the quantization step size from among the image blocks decoded in thepast. The predicted quantization step size generator 20113 generates apredicted quantization step size from a quantization step sizecorresponding to the image block selected. A difference quantizationstep size output from the entropy decoder 20111 is added to thegenerated predicted quantization step size, and the result is not onlyoutput as the quantization step size, but also input to the quantizationstep size buffer 20112.

Since the derivation method for the predicted quantization step size atthe predicted quantization step size generator 20113 is the same as thegeneration method for the predicted quantization step size at thepredicted quantization step size generator 10313 in the video encodingdevice in the third exemplary embodiment mentioned above, redundantdescription is omitted here.

Such a structure enables the video decoding device to decode thequantization step size by receiving only a further smaller code ratecompared with the video decoding device in the second exemplaryembodiment. As a result, a high-quality moving image can be decoded andregenerated. The reason is that the quantization step size can bepredicted from neighboring image blocks having higher correlation withthe image block concerned because the predicted quantization step sizegenerator 20113 predicts the quantization step size using the imageprediction parameter.

Example 1

Using an example, a specific operation of the quantization step sizeencoder in the video encoding device in the third exemplary embodimentmentioned above is described below.

In the example, the prediction direction of intra-frame prediction isused as the image prediction parameter to be used for prediction of thequantization step size. Further, as the intra-frame prediction,directional prediction of eight directions and average prediction(illustrated in FIG. 6) used for 4×4 pixel blocks and 8×8 pixel blocksin AVC described in NPL 1 are used.

It is assumed that the image block size as the unit of encoding is afixed size. It is also assumed that the block as the unit of determiningthe quantization step size (called quantization step size transmissionblock) and the block as the unit of intra-frame prediction (called aprediction block) are of the same size. If the current image block to beencoded is denoted by X, and four neighborhood blocks A, B, C, and Dhave a positional relationship shown in FIG. 2, the predictedquantization step size generator 10313 determines a predictedquantization step size pQ(X) by the following equation (6).pQ(X)=pQ(B); if m=0pQ(X)=pQ(A); if m=1pQ(X)=(pQ(A)+pQ(B)+1)/2; if m=2pQ(X)=pQ(C); if m=3pQ(X)=pQ(D); if m=4pQ(X)=(pQ(C)+pQ(D)+1)/2; if m=5pQ(X)=(pQ(A)+pQ(D)+1)/2; if m=6pQ(X)=(pQ(B)+pQ(D)+1)/2; if m=7pQ(X)=pQ(A); if m=8   (6)Note that m is an intra-prediction direction index in a frame shown inFIG. 6.

The entropy encoder 10312 applies the quantization step size Q(X) andthe predicted quantization step size pQ(X) to equation (4) to obtain adifference quantization step size dQ(X). The entropy encoder 10312encodes the obtained difference quantization step size dQ(X) using thesigned Exp-Golomb code as one of entropy codes, and outputs the resultas code corresponding to a quantization step size for the image blockconcerned.

In the example, directional prediction of eight directions and averageprediction are used as intra-frame prediction, but the present inventionis not limited thereto. For example, directional prediction of 33directions described in NPL 2 and average prediction may be used, or anyother intra-frame prediction may be used.

Further, the number of image blocks used for prediction may be anynumber other than four. In the example, as shown in the equation (6)mentioned above, either a quantization step size in any one of imageblocks or an average value of quantization step sizes in two imageblocks is used as the predicted quantization step size. However, thepresent invention is not limited to equation (6) mentioned above, andany other calculation result may be used as the predicted quantizationstep size. For example, as shown in the following equation (7), either aquantization step size in any one of image blocks or an intermediatevalue of three quantization step sizes may be used, or the predictedquantization step size may be determined using any other calculation.Further, the image blocks used for prediction are not necessarily to beadjacent to the current image block to be encoded. The image blocks usedfor prediction may be separated by a predetermined distance from thecurrent image block to be encoded.pQ(X)=pQ(B); if m=0,5 or 7pQ(X)=pQ(A); if m=1,6 or 8pQ(X)=pQ(C); if m=3pQ(X)=pQ(D); if m=4pQ(X)=Median(pQ(A),pQ(B),pQ(C)); if m=2   (7)

In the example, it is assumed that the image block to be encoded theneighboring image blocks are of the same fixed size. However, thepresent invention is not limited to the fixed size, and the block as theunit of encoding may be of a variable size.

Further, in the example, it is assumed that the quantization step sizetransmission blocks and the prediction block are of the same size.However, the present invention is not limited to the same size, and thequantization step size transmission blocks and the prediction block maybe of different sizes. For example, if two or more prediction blocks areincluded in the quantization step size transmission blocks, a predictionblock in any one of the two or more prediction blocks may be used forprediction of the quantization step size. Alternatively, the result ofadding any calculation, such as an intermediate value calculation or anaverage value calculation, to the prediction directions of the two ormore prediction blocks may be used for prediction of the quantizationstep size.

Further, in the example, the difference between the quantization stepsize of the image block to be encoded and the predicted quantizationstep size is encoded based on the Exp-Golomb code. However, the presentinvention is not limited to use of the Exp-Golomb code, and encodingbased on any other entropy code may be performed. For example, encodingbased on Huffman code or arithmetic code may be performed.

Example 2

Using another example, a specific operation of the quantization stepsize encoder in the video encoding device in the third exemplaryembodiment mentioned above is described below.

In this example, a motion vector of inter-frame prediction is used asthe image prediction parameter used for prediction of the quantizationstep size. Prediction defined by the translation of block units as shownin FIG. 7 is assumed as inter-frame prediction. It is assumed that aprediction image is generated from an image block located in a positionwhich is out of the same spatial position as the block to be encoded inthe reference frame by a displacement corresponding to the motionvector. Also, as shown in FIG. 7, prediction from a single referenceframe, i.e. one-directional prediction is assumed as inter-frameprediction. Further, in the example, it is assumed that the quantizationstep size transmission blocks and the prediction block are of the samesize.

Here, the block to be encoded is denoted by X, the center position ofblock X is denoted by cent(X), the motion vector in inter-frameprediction of X is denoted by V(X), and the reference frame to bereferred to in inter-frame prediction is denoted by RefPic(X). Then, asshow in FIG. 8, a block to which the position cent(X)+V(X) belongs inthe frame RefPic(X) is expressed as Block(RefPic(X),cent(X)+V(X)). Thepredicted quantization step size generator 10313 determines thepredicted quantization step size pQ(X) by the following equation (8).pQ(X)=Q(Block(RefPic(X),cent(X)+V(X))   (8)

The derivation of dQ(X) and the encoding process by the entropy encoder10312 are the same as those in the first example.

In the example, one-directional prediction is assumed, but the presentinvention is not limited to use of one-directional prediction. Forexample, in the case of bi-directional prediction, where a predictionimage is generated by weighted averaging reference image blocks in tworeference frames, if one reference frame is denoted by RefPic0(X), amotion vector to RefPic0(X) is denoted by V0(X), the other referenceframe is denoted by RefPic1(X), a motion vector to RefPic1(X) is denotedby V1(X), a weight given to RefPic0(X) upon generation of the predictionimage is denoted by w0, and a weight given to RefPic1(X) is denoted byw1, the predicted quantization step size generator 10313 may determinethe predicted quantization step size pQ(X) by the following equation(9).pQ(X)=w0Q(Block(RefPic0(X),cent(X)+V0(X))+w1Q(Block(RefPic1(X),cent(X)+V1(X))  (9)

Further, in the example, the quantization step size of the block towhich the center position of the reference image block belongs is usedas the predicted quantization step size, but the predicted quantizationstep size is not limited thereto. For example, a quantization step sizeof a block to which an upper left position of the reference image blockbelongs may be used as the predicted quantization step size.Alternatively, quantization step sizes of blocks to which all pixels ofthe reference image block belong may be respectively referred to use anaverage value of these quantization step sizes as the predictedquantization step size.

Further, in the example, prediction represented by the translationbetween blocks is assumed as inter-frame prediction. However, thereference image block is not limited thereto, and it may be of anyshape.

Further, in the example, it is assumed that the quantization step sizetransmission blocks and the prediction block are of the same size.However, like in the first example of the video encoding device in thethird exemplary embodiment mentioned above, the quantization step sizetransmission blocks and the prediction block may be of sizes differentfrom each other.

Example 3

Using still another example, a specific operation of the quantizationstep size encoder in the video encoding device in the third exemplaryembodiment mentioned above is described below.

In the example, prediction of a motion vector of inter-frame prediction,i.e. a predicted motion vector is used as the image prediction parameterused for prediction of the quantization step size. When the predictedmotion vector is derived from neighboring image blocks of the block tobe encoded, quantization step sizes of the neighboring image blocks usedfor derivation of the predicted motion vector is used to predict amotion vector of the block to be encoded.

In the example, it is assumed that the quantization step sizetransmission blocks and the prediction block are of the same size. Also,like in the second example of the video encoding device in the thirdexemplary embodiment mentioned above, one-directional predictionrepresented by a motion vector is assumed as inter-frame prediction. Inthe example, a predicted motion vector derived by a predetermined methodis subtracted from the motion vector shown in FIG. 7, and the differenceis entropy-encoded. As the predetermined predicted motion vectorderivation method, the predicted motion vector derivation methoddescribed in “8.4.2.1.4 Derivation process for luma motion vectorprediction” of NPL 2 is used.

Here, the predicted motion vector derivation method used in the exampleis briefly described. The block to be encoded is denoted by X, andblocks adjacent leftwardly, upwardly, diagonally right upward,diagonally left upward, and diagonally left downward as shown in FIG. 2are denoted by A, B, C, D, and E, respectively. A motion vector of blockA is denoted by mvA and a motion vector of block B is denoted by mvB.When block C exists in the image and has already been encoded, a motionvector of block C is set as mvC. Otherwise, when block D exists in theimage and has already been encoded, a motion vector of block D is set asmvC. Otherwise, a motion vector of block E is set as mvC.

Further, a motion vector determined by the following equation (10) isdenoted by mvMed, and a motion vector of a block in the same spatialposition as the block to be encoded on a reference frame assigned to theimage frame to be encoded (illustrated as an in-phase block XCol withrespect to the block X to be encoded in FIG. 8) is denoted by mvCol. Theassigned reference frame means, for example, an image frame encoded justbefore the image frame to be encoded.mvMed=(mvMedx,mvMedy)mvMedx=Median(mvAx,mvBx,mvCx)mvMedy=Median(mvAy,mvBy,mvCy)   (10)

As described above, five motion vectors, i.e. mvMed, mvA, mvB, mvC, andmvCol are candidates for the predicted motion vector in the block X tobe encoded. Any one motion vector is selected according to apredetermined priority order from among the candidates, and set as thepredicted motion vector pMV(X) of the block to be encoded. An example ofthe predetermined priority order is described in “8.4.2.1.4 Derivationprocess for luma motion vector prediction” and “8.4.2.1.8 Removalprocess for motion vector prediction” of NPL 2.

When the predicted motion vector pMV(X) is determined as mentionedabove, the predicted quantization step size generator 10313 determines apredicted quantization step size pQ(X) of the block X to be encoded bythe following equation (11).pQ(X)=Q(A); if pMV(X)=mvApQ(X)=Q(B); otherwise if pMV(X)=mvBpQ(X)=Q(C); otherwise if pMV(X)=mvC, and mvC is motion vector of blockCpQ(X)=Q(D); otherwise if pMV(X)=mvC, and mvC is motion vector of blockDpQ(X)=Q(E); otherwise if pMV(X)=mvC, and mvC is motion vector of blockEpQ(X)=Q(XCol); otherwise if pMV(X)=mvColpQ(X)=Median(Q(A),Q(B),Q(C)); otherwise   (11)

In the example, one-directional prediction is assumed, but the presentinvention is not limited to use of one-directional prediction. Like inthe second example of the video encoding device in the third exemplaryembodiment mentioned above, this example can also be applied tobi-directional prediction.

Further, in the example, the predicted motion vector derivation methoddescribed in “8.4.2.1.4 Derivation process for luma motion vectorprediction” of NPL 2 is used as the predicted motion vector derivationmethod, but the present invention is not limited thereto. For example,as described in “8.4.2.1.3 Derivation process for luma motion vectorsfor merge mode” of NPL 2, if the motion vector of the block X to beencoded is predicted by a motion vector of either block A or block B,the predicted quantization step size generator 10313 may determine thepredicted quantization step size pQ(X) of the block X to be encoded bythe following equation (12), or any other predicted motion vectorderivation method may be used.pQ(X)=Q(A); if pMV(X)=mvApQ(X)=Q(B); otherwise   (12)

Further, in the example, the image blocks used for prediction of thequantization step size are referred to as shown in equation (11) inorder of blocks A, B, C, D, E, and XCol. However, the present inventionis not limited to this order, and any order may be used. As for thenumber and positions of image blocks used for prediction of thequantization step size, any number and positions of image blocks may beused. Further, in the example, an intermediate value calculation like inequation (3) is used when pMV(X) agrees with none of mvA, mvB, mvC, andmvCol, but the present invention is not limited to use of theintermediate value calculation. Any calculation such as the averagevalue calculation like in the first exemplary embodiment may also beused.

Further, in the example, it is assumed that the quantization step sizetransmission blocks and the prediction block are of the same size.However, the quantization step size transmission blocks and theprediction block may be of sizes different from each other like in thefirst example and second example of the video encoding device in thethird exemplary embodiment mentioned above.

Exemplary Embodiment 5

FIG. 9 is a block diagram showing the structure of a video encodingdevice in a fifth exemplary embodiment of the present invention. FIG. 10is a block diagram showing the structure of a quantization step sizeencoder in the video encoding device in this exemplary embodiment.

In comparison with the video encoding device shown in FIG. 24, the videoencoding device in this exemplary embodiment is different in that aquantization step size prediction controller 111 and a multiplexer 112are included as shown in FIG. 9. Note that the video encoding deviceshown in FIG. 24 is also the video encoding device in the thirdexemplary embodiment as described above.

Further, as shown in FIG. 10, this exemplary embodiment differs fromthird exemplary embodiment in that a quantization step size encoder forencoding the quantization step size in the variable-length encoder 103of the video encoding device is configured to supply the quantizationstep size prediction parameter from the quantization step sizeprediction controller 111 shown in FIG. 9 to the predicted quantizationstep size generator 10313 in comparison with the quantization step sizeencoder shown in FIG. 4, and in the operation of the predictedquantization step size generator 10313.

The quantization step size prediction controller 111 supplies controlinformation for controlling the quantization step size predictionoperation of the predicted quantization step size generator 10313 to thevariable-length encoder 103 and the multiplexer 112. The controlinformation for controlling the quantization step size predictionoperation is called a quantization step size prediction parameter.

The multiplexer 112 multiplexes the quantization step size predictionparameter into a video bitstream supplied from the variable-lengthencoder 103, and outputs the result as a bitstream.

Using the image prediction parameter and the quantization step sizeprediction parameter, the predicted quantization step size generator10313 selects an image block used for prediction of the quantizationstep size from among image blocks encoded in the past. The predictedquantization step size 10313 also generates a predicted quantizationstep size from a quantization step size corresponding to the image blockselected.

Such a structure of the video encoding device in the exemplaryembodiment can further reduce the code rate required to encode thequantization step size in comparison with the video encoding device inthe third exemplary embodiment. As a result, high-quality moving imageencoding can be achieved. The reason is that the quantization step sizecan be predicted for the image block with a higher accuracy, because thepredicted quantization step size generator 10313 uses the quantizationstep size prediction parameter in addition to the image predictionparameter to switch or correct a prediction value of the quantizationstep size using the image prediction parameter. The reason why thequantization step size can be predicted with a higher accuracy byswitching or correction using the quantization step size predictionparameter is because the quantization controller 104 shown in FIG. 9monitors the output code rate of the variable-length encoder 103 toincrease or decrease the quantization step size without depending on thehuman visual sensitivity alone, and hence a quantization step size to begiven also to image blocks having the same visual sensitivity can vary.

A specific operation of the quantization step size encoder in the videoencoding device in the fifth exemplary embodiment mentioned above isdescribed using a specific example below.

In this example, like in the second example of the video encoding devicein the third exemplary embodiment mentioned above, a motion vector ofinter-frame prediction is used as the image prediction parameter usedfor prediction of the quantization step size. Prediction defined by thetranslation of block units as shown in FIG. 7 is assumed as inter-frameprediction. In this case, it is assumed that a prediction image isgenerated from an image block located in a position which is out of thesame spatial position as the block to be encoded in the reference frameby a displacement corresponding to the motion vector. Also, as shown inFIG. 7, prediction from a single reference frame, i.e. one-directionalprediction is assumed as inter-frame prediction. Further, in theexample, it is assumed that the quantization step size transmissionblocks and the prediction block are of the same size.

Here, the block to be encoded is denoted by X, the frame to be encodedis denoted by Pic(X), the center position of block X is denoted bycent(X), the motion vector in inter-frame prediction of X is denoted byV(X), and the reference frame to be referred to in inter-frameprediction is denoted by RefPic(X). Then, as show in FIG. 8, a block towhich the position cent(X)+V(X) belongs in the frame RefPic(X) isexpressed as Block(RefPic(X),cent(X)+V(X)). Further, it is assumed thatthree neighboring image blocks A, B, and C are located in positionsrespectively adjacent leftwardly, upwardly, and diagonally right upwardto block X as shown in FIG. 2. In this case, the predicted quantizationstep size generator 10313 determines the predicted quantization stepsize pQ(X) by the following equation (13).pQ(X)=Q(Block(RefPic(X),cent(X)+V(X)); if temporal_qp_pred_flag=1pQ(X)=Median(pQ(A),pQ(B),Q(C)); otherwise   (13)

Here, temporal_qp_pred_flag represents a flag for switching betweenwhether or not the motion vector between frames can be used forprediction of the quantization step size. The flag is supplied from thequantization step size prediction controller 111 to the predictedquantization step size generator 10313.

The predicted quantization step size generator 10313 may also use anoffset value for compensating for a change in quantization step sizebetween the frame Pic(X) to be encoded and the reference frameRefPic(X), i.e. an offset to the quantization step size Qofs(Pic(X),RefPic(X)) to determine the predicted quantization step size pQ(X) bythe following equation (14).pQ(X)=Q(Block(RefPic(X),cent(X)+V(X))+Qofs(Pic(X),RefPic(X))  (14)

Further, the predicted quantization step size generator 10313 may useboth temporal_qp_pred_flag mentioned above and the offset to thequantization step size to determine the predicted quantization step sizepQ(X) by the following equation (15).pQ(X)=Q(Block(RefPic(X),cent(X)+V(X))+Qofs(Pic(X),RefPic(X)); iftemporal_qp_pred_flag=1pQ(X)=Median(pQ(A),pQ(B),Q(C)); otherwise   (15)

For example, if the initial quantization step size of any frame Z isdenoted by Qinit(Z), the offset to the quantization step sizeQofs(Pic(X), RefPic(X)) in equations (14) and (15) mentioned above maybe determined by the following equation (16).Qofs(Pic(X),RefPic(X))=Qinit(Pic(X))−Qinit(RefPic(X))  (16)

The initial quantization step size is a value given as the initial valueof the quantization step size for each frame, and SliceQP_(Y) describedin “7.4.3 Slice header semantics” of NPL 1 may be used, for example.

For example, as illustrated in a list shown in FIG. 11, whichcorresponds to the description in “Specification of syntax functions,categories, and descriptors” of NPL 1, either or both of thetemporal_qp_pred_flag value and the Qofs(Pic(X), RefPic(X)) valuementioned above may be multiplexed into a bitstream as part of headerinformation.

In the list shown in FIG. 11, qp_pred_offset represents the Qofs valuein equation (14) mentioned above. As shown in FIG. 11, multiple piecesof qp_pred_offset may be multiplexed as Qofs values corresponding torespective reference frames, or one piece of qp_pred_offset may bemultiplexed as a common Qofs value to all the reference frames.

In the example, the motion vector of inter-frame prediction is assumedas the image prediction parameter. However, the present invention is notlimited to use of the motion vector of inter-frame prediction. Like inthe first example of the video encoding device in the third exemplaryembodiment mentioned above, the prediction direction of intra-frameprediction may be so used that the flag mentioned above will switchbetween whether to use the prediction direction of intra-frameprediction or not for prediction of the quantization step size. Like inthe third example of the video encoding device in the third exemplaryembodiment mentioned above, the prediction direction of the predictedmotion vector may be used, or any other image prediction parameter maybe used.

Further, in the example, one-directional prediction is assumed asinter-frame prediction. However, the present invention is not limited touse of one-directional prediction. Like in the second example of thevideo encoding device in the third exemplary embodiment mentioned above,the present invention can also be applied to bi-directional prediction.

Further, in the example, the quantization step size of a block to whichthe center position of the reference image block belongs is used as thepredicted quantization step size. However, the derivation of thepredicted quantization step size in the present invention is not limitedthereto. For example, the quantization step size of a block to which theupper left position of the reference image block belongs may be used asthe predicted quantization step size. Alternatively, quantization stepsizes of blocks to which all pixels of the reference image block belongmay be respectively referred to use an average value of thesequantization step sizes as the predicted quantization step size.

Further, in the example, prediction represented by the translationbetween blocks of the same shape is assumed as inter-frame prediction.However, the reference image block in the present invention is notlimited thereto, and it may be of any shape.

Further, in the example, as shown in equation (13) and equation (15),when inter-frame prediction information is not used, the quantizationstep size is predicted from three spatially neighboring image blocksbased on the intermediate value calculation, but the present inventionis not limited thereto. Like in the specific example of the firstexemplary embodiment, the number of image blocks used for prediction maybe any number other than three, and an average value calculation or thelike may be used instead of the intermediate value calculation. Further,the image blocks used for prediction are not necessarily to be adjacentto the current image block to be encoded, and the image blocks may beseparated by a predetermined distance from the current image block to beencoded.

Further, in the example, it is assumed that the quantization step sizetransmission blocks and the prediction block are of the same size, butlike in the first example of the video encoding device in the thirdexemplary embodiment mentioned above, the quantization step sizetransmission blocks and the prediction block may be of sizes differentfrom each other.

Exemplary Embodiment 6

FIG. 12 is a block diagram showing the structure of a video decodingdevice in a sixth exemplary embodiment of the present invention. FIG. 13is a block diagram showing the structure of a quantization step sizedecoder in the video decoding device in the exemplary embodiment.

In comparison with the video decoding device shown in FIG. 26, the videodecoding device in the exemplary embodiment differs in including ade-multiplexer 208 and a quantization step size prediction controller209 as shown in FIG. 12. As described above, the video decoding deviceshown in FIG. 26 is also the video decoding device in the fourthexemplary embodiment.

Further, in comparison with the quantization step size decoder shown inFIG. 5, a quantization step size decoder for decoding the quantizationstep size in the variable-length decoder 201 of the video decodingdevice in the exemplary embodiment differs as shown in FIG. 13 from thefourth exemplary embodiment in that the quantization step sizeprediction parameter is supplied from the quantization step sizeprediction controller 209 shown in FIG. 12 to the predicted quantizationstep size generator 20113, an in the operation of the predictedquantization step size generator 20113.

The de-multiplexer 208 de-multiplexes a bitstream to extract a videobitstream and control information for controlling the quantization stepsize prediction operation. The de-multiplexer 208 further supplies theextracted control information to the quantization step size predictioncontroller 209, and the extracted video bitstream to the variable-lengthdecoder 201, respectively.

The quantization step size prediction controller 209 sets up theoperation of the predicted quantization step size generator 20113 basedon the control information supplied.

The predicted quantization step size generator 20113 uses the imageprediction parameter and the quantization step size prediction parameterto select an image block used for prediction of the quantization stepsize from among the image blocks decoded in the past. The predictedquantization step size generator 20113 further generates a predictedquantization step size from a quantization step size corresponding tothe selected image block. A difference quantization step size outputfrom the entropy decoder 20111 is added to the generated predictedquantization step size, and the result is not only output as thequantization step size, but also input to the quantization step sizebuffer 20112.

Since the derivation method for the predicted quantization step size atthe predicted quantization step size generator 20113 is the same as thegeneration method for the predicted quantization step size at thepredicted quantization step size generator 10313 in the video encodingdevice in the fifth exemplary embodiment mentioned above, redundantdescription is omitted here.

Such a structure enables the video decoding device to decode thequantization step size by receiving only a further smaller code ratecompared with the video decoding device in the fourth exemplaryembodiment. As a result, a high-quality moving image can be decoded andregenerated. The reason is that the quantization step size can bepredicted for the image block with a higher accuracy because thepredicted quantization step size generator 20113 uses the quantizationstep size prediction parameter in addition to the image predictionparameter to switch or correct a predicted value of the quantizationstep size using the image prediction parameter.

Exemplary Embodiment 7

Like the video encoding device in the third exemplary embodiment, avideo encoding device in a seventh exemplary embodiment of the presentinvention includes the frequency transformer 101, the quantizer 102, thevariable-length encoder 103, the quantization controller 104, theinverse quantizer 105, the inverse frequency transformer 106, the framememory 107, the intra-frame predictor 108, the inter-frame predictor109, and the prediction selector 110 as shown in FIG. 24. However, thestructure of a quantization step size encoder included in thevariable-length encoder 103 is different from the structure of the videoencoding device in the third exemplary embodiment shown in FIG. 4.

FIG. 14 is a block diagram showing the structure of the quantizationstep size encoder in the video encoding device in the seventh exemplaryembodiment of the present invention. In comparison with the quantizationstep size encoder shown in FIG. 4, the structure of the quantizationstep size encoder in the exemplary embodiment is different in includinga quantization step size selector 10314 as shown in FIG. 14.

Since the operation of the quantization step size buffer 10311, theentropy encoder 10312, and the predicted quantization step sizegenerator 10313 is the same as the operation of the quantization stepsize encoder in the video encoding device in the third exemplaryembodiment, redundant description is omitted here.

The quantization step size selector 10314 selects either a quantizationstep size assigned to the previously encoded image block or a predictedquantization step size output from the predicted quantization step sizegenerator 10313 according to the image prediction parameter, and outputsthe result as a selectively predicted quantization step size. Thequantization step size assigned to the previously encoded image block issaved in the quantization step size buffer 10311. The selectivelypredicted quantization step size output from the quantization step sizeselector 10314 is subtracted from the quantization step size input tothe quantization step size encoder and to be currently encoded, and theresult is input to the entropy encoder 10312.

Such a structure enables the video encoding device in the exemplaryembodiment to further reduce the code rate required to encode thequantization step size compared with the video encoding device in thethird exemplary embodiment. As a result, high-quality moving imageencoding can be achieved. The reason is that the quantization step sizecan be encoded by the operation of the quantization step size selector10314 to selectively use the predicted quantization step size derivedfrom the image prediction parameter and the previously encodedquantization step size. The reason why the code rate required to encodethe quantization step size can be further reduced by selectively usingthe predicted quantization step size derived from the image predictionparameter and the previously encoded quantization step size is becausethe quantization controller 104 in the encoding device not only performsvisual-sensitivity-based adaptive quantization but also monitors theoutput code rate to increase or decrease the quantization step size asdescribed above.

A specific operation of the quantization step size encoder in the videoencoding device in the seventh exemplary embodiment is described using aspecific example below.

Here, the prediction direction of intra-frame prediction is used as theimage prediction parameter used for prediction of the quantization stepsize. Further, as the intra-frame prediction, directional prediction ofeight directions and average prediction (see FIG. 6) used for 4×4 pixelblocks and 8×8 pixel blocks in the AVC scheme described in NPL 1 areused.

It is assumed that the image block size as the unit of encoding is afixed size. It is also assumed that the block as the unit of determiningthe quantization step size (called quantization step size transmissionblock) and the block as the unit of intra-frame prediction (called aprediction block) are of the same size. If the current image block to beencoded is denoted by X, and four neighborhood blocks A, B, C, and Dhave a positional relationship shown in FIG. 2, the predictedquantization step size generator 10313 determines the predictedquantization step size pQ(X) by equation (6) mentioned above.

The quantization step size selector 10314 selects either the predictedquantization step size pQ(X) obtained by equation (6) or the previouslyencoded quantization step size Q(Xprev) according to the followingequation (17) to generate a selectively predicted quantization step sizesQ(X), i.e. the predicted quantization step size determined by equation(6) is used as the selectively predicted quantization step size fordirectional prediction and the previous quantization step size is usedas the selectively predicted quantization step size for average valueprediction.sQ(X)=Q(Xprev); if m=2sQ(X)=pQ(X); if m=0,1,3,4,5,6,7 or 8   (17)Note that m is an intra-frame prediction direction index in the frameshown in FIG. 6.

The entropy encoder 10312 encodes a difference quantization step sizedQ(X) obtained by the following equation (18) using the signedExp-Golomb (Exponential-Golomb) code as one of entropy codes, andoutputs the result as code corresponding to a quantization step size forthe image block concerned.dQ(X)=Q(X)−sQ(X)  (18)

In the exemplary embodiment, direction prediction of eight directionsand average prediction are used as intra-frame prediction, but thepresent invention is not limited thereto. For example, directionalprediction of 33 directions described in NPL 2 and average predictionmay be used, or any other intra-frame prediction may be used.

Further, in the exemplary embodiment, the selection between thepredicted quantization step size and the previously encoded quantizationstep size is made based on the parameters of intra-frame prediction, butthe present invention is not limited to use of intra-frame predictioninformation. For example, selections may be made to use the predictedquantization step size in the intra-frame prediction block and thepreviously encoded quantization step size in the inter-frame predictionblock, or vice versa. When the parameters of inter-frame prediction meeta certain specific condition, a selection may be made to use thepreviously encoded quantization step size.

The number of image blocks used for prediction may be any number otherthan four. Further, in the exemplary embodiment, either a quantizationstep size in any one of image blocks or an average value of quantizationstep sizes in two image blocks is used as the predicted quantizationstep size as shown in equation (6). However, the predicted quantizationstep size is not limited to those in equation (6). Any other calculationresult may be used as the predicted quantization step size. For example,as shown in equation (7), either a quantization step size in any one ofimage blocks or an intermediate value of three quantization step sizesmay be used, or the predicted quantization step size may be determinedusing any other calculation. Further, the image blocks used forprediction are not necessarily to be adjacent to the current image blockto be encoded. The image blocks used for prediction may be separated bya predetermined distance from the current image block to be encoded.

Further, in the exemplary embodiment, it is assumed that the image blockto be encoded and the image blocks used for prediction are of the samefixed size. However, the present invention is not limited to the casewhere the image block as the unit of encoding is of a fixed size. Theimage block as the unit of encoding may be of a variable size, and theimage block to be encoded and the image blocks used for prediction maybe of sizes different from each other.

Further, in the exemplary embodiment, it is assumed that thequantization step size transmission blocks and the prediction block areof the same size. However, the present invention is not limited to thecase of the same size, and the quantization step size transmissionblocks and the prediction block may be of different sizes. For example,when two or more prediction blocks are included in the quantization stepsize transmission blocks, the prediction direction of any one predictionblock of the two or more prediction blocks may be used for prediction ofthe quantization step size. Alternatively, the result of adding anycalculation, such as the intermediate value calculation or the averagevalue calculation, to the prediction directions of the two or moreprediction blocks may be used for prediction of the quantization stepsize.

Further, in the exemplary embodiment, the difference between thequantization step size of the image block to be encoded and thepredicted quantization step size is encoded based on the Exp-Golombcode. However, the present invention is not limited to use of theExp-Golomb code, and encoding based on any other entropy code may beperformed. For example, encoding based on Huffman code or arithmeticcode may be performed.

Exemplary Embodiment 8

Like the video decoding device in the fourth exemplary embodiment of thepresent invention, a video decoding device in an eighth exemplaryembodiment of the present invention includes the variable-length decoder201, the inverse quantizer 202, the inverse frequency transformer 203,the frame memory 204, the intra-frame predictor 205, the inter-framepredictor 206, and the prediction selector 207 as shown in FIG. 26.However, the structure of a quantization step size decoder included inthe variable-length decoder 201 is different from the structure shown inFIG. 5.

FIG. 15 is a block diagram showing a quantization step size decoder inthe video decoding device in the eighth exemplary embodiment of thepresent invention. In comparison with the structure of the quantizationstep size decoder shown in FIG. 5, the structure of the quantizationstep size decoder in the exemplary embodiment is different in includinga quantization step size selector 20114 as shown in FIG. 15.

Since the operation of the entropy decoder 20111, the quantization stepsize buffer 20112, and the predicted quantization step size generator20113 is the same as the operation of the quantization step size decoderin the video encoding device in the fourth exemplary embodiment,redundant description is omitted here.

The quantization step size selector 20114 selects either a quantizationstep size assigned to the previously decoded image block or a predictedquantization step size output from the predicted quantization step sizegenerator 20113 according to the image prediction parameter, and outputsthe result as a selectively predicted quantization step size. Thequantization step size assigned to the previously decoded image block issaved in the quantization step size buffer 20112. A differencequantization step size generated by the entropy decoder 20111 is addedto the selectively predicted quantization step size output, and theresult is not only output as the quantization step size, but also storedin the quantization step size buffer 20112.

Such a structure enables the video decoding device to decode thequantization step size by receiving only a further smaller code ratecompared with the video decoding device in the fourth exemplaryembodiment. As a result, a high-quality moving image can be decoded andregenerated. The reason is that the quantization step size can bedecoded by the operation of the quantization step size selector 20114 toselectively use the predicted quantization step size derived from theimage prediction parameter and the previously encoded quantization stepsize so that the quantization step size can be decoded with a smallercode rate for a bitstream generated by applying both thevisual-sensitivity-based adaptive quantization and the increase ordecrease in quantization step size resulting from monitoring the outputcode rate, and hence a moving image can be decoded and regenerated bythe smaller code rate.

Each of the exemplary embodiments mentioned above may be realized byhardware, or a computer program.

An information processing system shown in FIG. 16 includes a processor1001, a program memory 1002, a storage medium 1003 for storing videdata, and a storage medium 1004 for storing a bitstream. The storagemedium 1003 and the storage medium 1004 may be separate storage media,or storage areas included in the same storage medium. As the storagemedium, a magnetic storage medium such as a hard disk can be used as thestorage medium.

In the information processing system shown in FIG. 16, a program forimplementing the function of each block (including each of the blocksshown in FIG. 1, FIG. 3, FIG. 4, and FIG. 5, except the buffer block)shown in each of FIG. 24 and FIG. 26 is stored in the program memory1002. The processor 1001 performs processing according to the programstored in the program memory 1002 to implement the functions of thevideo encoding device or the video decoding device shown in each of FIG.24, FIG. 26, and FIG. 1, FIG. 3, FIG. 4, and FIG. 5, respectively.

FIG. 17 is a block diagram showing characteristic components in a videoencoding device according to the present invention. As shown in FIG. 17,the video encoding device according to the present invention includes aquantization step size encoding unit 10 for encoding a quantization stepsize that controls the granularity of quantization, and the quantizationstep size encoding unit 10 includes a quantization step size predictionunit 11 for predicting the quantization step size using quantizationstep sizes assigned to neighboring image blocks already encoded.

FIG. 18 is a block diagram showing characteristic components in anothervideo encoding device according to the present invention. As shown inFIG. 18, the other video encoding device according to the presentinvention includes, in addition to the structure shown in FIG. 17, aprediction image generation unit 20 for using images encoded in the pastand a predetermined parameter to generate a prediction image of an imageblock to be encoded. In this structure, the quantization step sizeencoding unit 10 predicts the quantization step size using parametersused in generating the prediction image. A predicted motion vectorgeneration unit 30 for predicting a motion vector used for inter-frameprediction by using motion vectors assigned to neighboring image blocksalready encoded may also be included so that the quantization step sizeencoding unit 10 will use a prediction direction of the predicted motionvector to predict the quantization step size.

FIG. 19 is a block diagram showing characteristic components in a videodecoding device according to the present invention. As shown in FIG. 19,the video decoding device according to the present invention includes aquantization step size decoding unit 50 for decoding a quantization stepsize that controls the granularity of inverse quantization, and thequantization step size decoding unit 50 includes a step size predictionunit 51 for predicting the quantization step size using quantizationstep sizes assigned to neighboring image blocks already decoded.

FIG. 20 is a block diagram showing characteristic components in anothervideo decoding device according to the present invention. As shown inFIG. 20, the other video decoding device according to the presentinvention includes, in addition to the structure shown in FIG. 19, aprediction image generation unit 60 for using images decoded in the pastand predetermined parameters to generate a prediction image of an imageblock to be decoded. In this structure, the quantization step sizedecoding unit 50 predicts a quantization step size using parameters usedin generating the prediction image. A predicted motion vector generationunit 70 for predicting a motion vector used for inter-frame predictionby using a motion vector assigned to a neighboring image block alreadyencoded may also be so included that the quantization step size decodingunit 50 will use a prediction direction of the predicted motion vectorto predict the quantization step size.

FIG. 21 is a flowchart showing characteristic steps in a video encodingmethod according to the present invention. As shown in FIG. 21, thevideo encoding method includes step S11 for determining a predictiondirection of intra-frame prediction, step S12 for generating aprediction image using intra-frame prediction, and step S13 forpredicting a quantization step size using the prediction direction ofintra-frame prediction.

FIG. 22 is a flowchart showing characteristic steps in a video decodingmethod according to the present invention. As shown in FIG. 22, thevideo decoding method includes step S21 for determining a predictiondirection of intra-frame prediction, step S22 for generating aprediction image using intra-frame prediction, and step S23 forpredicting a quantization step size using the prediction direction ofintra-frame prediction.

Part or all of the aforementioned exemplary embodiments can be describedas Supplementary notes mentioned below, but the structure of the presentinvention is not limited to the following structures.

(Supplementary Note 1)

A video encoding device for dividing input image data into blocks of apredetermined size, and applying quantization to each divided imageblock to execute a compressive encoding process, comprising quantizationstep size encoding means for encoding a quantization step size thatcontrols the granularity of quantization, and prediction imagegeneration means for using an image encoded in the past and apredetermined parameter to generate a prediction image of an image blockto be encoded, the quantization step size encoding means for predictingthe quantization step size by using the parameter used by the predictionimage generation means, wherein the prediction image generation meansgenerates the prediction image by using at least inter-frame prediction,and the quantization step size encoding means uses a motion vector ofthe inter-frame prediction to predict the quantization step size.

(Supplementary Note 2)

A video encoding device for dividing input image data into blocks of apredetermined size, and applying quantization to each divided imageblock to execute a compressive encoding process, comprising quantizationstep size encoding means for encoding a quantization step size thatcontrols the granularity of quantization, and prediction imagegeneration means for generating a prediction image of an image block tobe encoded by using an image encoded in the past and a predeterminedparameter, the quantization step size encoding means for predicting thequantization step size by using the parameter used by the predictionimage generation means, wherein the quantization step size encodingmeans predicts the quantization step size by using a quantization stepsize assigned to a neighboring image block already encoded, theprediction image generation means generates the prediction image byusing at least inter-frame prediction, predicted motion vectorgeneration means for predicting a motion vector used for inter-frameprediction by using a motion vector assigned to the neighboring imageblock already encoded is further comprised, and the quantization stepsize encoding means uses a prediction direction of the predicted motionvector to predict the quantization step size.

(Supplementary Note 3)

A video decoding device for decoding image blocks using inversequantization of input compressed video data to execute a process ofgenerating image data as a set of the image blocks, comprisingquantization step size decoding means for decoding a quantization stepsize that controls a granularity of inverse quantization, and predictionimage generation means for generating a prediction image of an imageblock to be decoded by using an image decoded in the past and apredetermined parameter, the quantization step size decoding means forpredicting the quantization step size by using the parameter assigned toa neighboring image block already decoded, wherein the quantization stepsize decoding means predicts the quantization step size by using theparameter used to generate the prediction image, the prediction imagegeneration means generates the prediction image by using at leastinter-frame prediction, and the quantization step size decoding meansuses a motion vector of the inter-frame prediction to predict thequantization step size.

(Supplementary Note 4)

A video decoding device for decoding image blocks using inversequantization of input compressed video data to execute a process ofgenerating image data as a set of the image blocks, comprisingquantization step size decoding means for decoding a quantization stepsize that controls the granularity of inverse quantization, andprediction image generation means for generating a prediction image ofan image block to be decoded by using an image decoded in the past and apredetermined parameter, the quantization step size decoding means forpredicting the quantization step size by using a quantization step sizeassigned to a neighboring image already decoded, wherein thequantization step size decoding means predicts the quantization stepsize using the prediction image used to generate the prediction image,the prediction image generation means generates the prediction imageusing at least inter-frame prediction, predicted motion vectorgeneration means for using a motion vector assigned to the neighboringimage block already encoded to predict a motion vector used forinter-frame prediction is further comprised, and the quantization stepsize decoding means uses a prediction direction of the predicted motionvector to predict the quantization step size.

(Supplementary Note 5)

A video encoding method for dividing input image data into blocks of apredetermined size, and applying quantization to each divided imageblock to execute a compressive encoding process, comprising a step ofpredicting a quantization step size that controls the granularity ofquantization using a quantization step size assigned to a neighboringimage block already encoded, and a step of to generating a predictionimage of an image block to be encoded by using an image encoded in thepast and a predetermined parameter, wherein the quantization step sizeis predicted by using the parameter used to generate the predictionimage.

(Supplementary Note 6)

The video encoding method according to Supplementary note 5, wherein theprediction image is generated using at least intra-frame prediction inthe step of generating the prediction image, and a prediction directionof the intra-frame prediction is used to predict the quantization stepsize.

(Supplementary Note 7)

The video encoding method according to Supplementary note 5, wherein theprediction image is generated using at least inter-frame prediction inthe step of generating the prediction image, and a motion vector of theinter-frame prediction is used to predict the quantization step size.

(Supplementary Note 8)

The video encoding method according to Supplementary note 5, theprediction image is generated using at least inter-frame prediction inthe step of generating the prediction image, a step of using a motionvector assigned to a neighboring image block already encoded to apredict a motion vector used for inter-frame prediction is comprised,and a prediction direction of the predicted motion vector is used topredict the quantization step size.

(Supplementary Note 9)

A video encoding method for decoding image blocks using inversequantization of input compressed video data to execute a process ofgenerating image data as a set of the image blocks, comprising a step ofpredicting a quantization step size that controls the granularity ofinverse quantization by using a quantization step size assigned to aneighboring image block already decoded, and a step of generating aprediction image using at least inter-frame prediction, wherein a motionvector of the inter-frame prediction is used to predict the quantizationstep size.

(Supplementary Note 10)

A video decoding method for decoding image blocks using inversequantization of input compressed video data to execute a process ofgenerating image data as a set of the image blocks, comprising a step ofpredicting a quantization step size that controls the granularity ofinverse quantization by using a quantization step size assigned to aneighboring image block already decoded, and a step of generating aprediction image using at least inter-frame prediction, a motion vectorassigned to a neighboring image block already encoded is used to predicta motion vector is used for inter-frame prediction, and a predictiondirection of the predicted motion vector is used to predict thequantization step size.

(Supplementary Note 11)

A video encoding program used in a video encoding device for dividinginput image data into blocks of a predetermined size, and applyingquantization to each divided image block to execute a compressiveencoding process, causing a computer to use a quantization step sizeassigned to a neighboring image block already encoded in order topredict a quantization step size that controls the granularity ofquantization.

(Supplementary Note 12)

The video encoding program according to Supplementary note (11), causingthe computer to use an image encoded in the past and a predeterminedparameter to execute a process of generating a prediction image of animage block to be encoded in order to predict the quantization step sizeusing the parameter used to generate the prediction image.

(Supplementary Note 13)

The video encoding program according to Supplementary note (12), causingthe computer to execute the process of generating the prediction imageusing at least intra-frame prediction in order to predict thequantization step size using a prediction direction of the intra-frameprediction.

(Supplementary Note 14)

The video encoding program according to Supplementary note (12), causingthe computer to execute the process of generating the prediction imageusing at least inter-frame prediction in order to predict thequantization step size using a motion vector of the inter-frameprediction.

(Supplementary Note 15)

The video encoding program according to Supplementary note (12), causingthe computer to execute the process of generating the prediction imageusing at least inter-frame prediction and a process of using a motionvector assigned to a neighboring image block already encoded to predicta motion vector used in inter-frame prediction in order to predict thequantization step size using a prediction direction of the predictedmotion vector.

(Supplementary Note 16)

A video decoding program used in a video decoding device for decodingimage blocks using inverse quantization of input compressed video datato execute a process of generating image data as a set of the imageblocks, causing a computer to use a quantization step size assigned to aneighboring image block already decoded in order to predict aquantization step size that controls the granularity of inversequantization.

(Supplementary Note 17)

The video decoding program according to Supplementary note (16), causingthe computer to execute a process of using an image decoded in the pastand a predetermined parameter to generate a prediction image of an imageblock to be decoded in order to predict the quantization step size usingthe parameter used to generate the prediction image.

(Supplementary Note 18)

The video decoding program according to Supplementary note (17), causingthe computer to execute the process of generating the prediction imageusing at least intra-frame prediction in order to predict thequantization step size using a prediction direction of the intra-frameprediction.

(Supplementary Note 19)

The video decoding program according to Supplementary note (17), causingthe computer to execute the process of generating the prediction imageusing at least inter-frame prediction in order to predict thequantization step size using a motion vector of the inter-frameprediction.

(Supplementary Note 20)

The video decoding program according to Supplementary note (17), causingthe computer to execute the process of generating the prediction imageusing at least inter-frame prediction and a process of using a motionvector assigned to a neighboring image block already encoded to predicta motion vector used in inter-frame prediction in order to predict thequantization step size using a prediction direction of the predictedmotion vector.

(Supplementary Note 21)

A video encoding device for dividing input image data into blocks of apredetermined size, and applying quantization to each divided imageblock to execute a compressive encoding process, comprising quantizationstep size encoding means for encoding a quantization step size thatcontrols the granularity of quantization; prediction image generationmeans for generating a prediction image of an image block to be encodedby using an image encoded in the past and a predetermined parameter,wherein the quantization step size encoding means predicts thequantization step size using the parameter used by the prediction imagegeneration means; quantization step size prediction control means forcontrolling the operation of the quantization step size encoding meansbased on the predetermined parameter; and multiplexing means formultiplexing an operational parameter of the quantization step sizeencoding means into the result of the compressive encoding process.

(Supplementary Note 22)

The video encoding device according to Supplementary note 21, whereinthe operational parameter of the quantization step size encoding meansincludes at least a flag representing whether to use the parameter usedby the prediction image generation means or not, and the quantizationstep size prediction control means controls the operation of thequantization step size encoding means based on the flag.

(Supplementary Note 23)

The video encoding device according to Supplementary note 21, whereinthe operational parameter of the quantization step size encoding meanscomprises at least a modulation parameter of the quantization step size,and the quantization step size encoding means uses the modulationparameter to modulate the quantization step size determined based on theparameter used by the prediction image generation means in order topredict the quantization step size.

(Supplementary Note 24)

The video encoding device according to Supplementary note 23, whereinthe quantization step size encoding means adds a predetermined offset tothe quantization step size determined based on the parameter used by theprediction image generation means in order to predict the quantizationstep size.

(Supplementary Note 25)

A video decoding device for decoding image blocks using inversequantization of input compressed video data to execute a process ofgenerating image data as a set of the image blocks, comprising:quantization step size decoding means for decoding a quantization stepsize that controls the granularity of inverse quantization; predictionimage generation means for using an image decoded in the past and apredetermined parameter to generate a prediction image of an image blockto be decoded wherein the quantization step size decoding means uses aquantization step size assigned to a neighboring image block alreadydecoded to predict the quantization step size; de-multiplexing mean forde-multiplexing a bitstream including an operational parameter of thequantization step size decoding means; and quantization step sizeprediction control means for controlling the operation of thequantization step size decoding means based on the de-multiplexedoperational parameter of the quantization step size decoding means.

(Supplementary Note 26)

The video decoding device according to Supplementary note 25, whereinthe de-multiplexing means extracts, as the operational parameter of thequantization step size decoding means, at least a flag representingwhether to use the parameter used by the prediction image generationmeans, and the quantization step size prediction control means controlsthe operation of the quantization step size decoding means based on theflag.

(Supplementary Note 27)

The video decoding device according to Supplementary note 25, whereinthe de-multiplexing means extracts, as the operational parameter of thequantization step size decoding means, at least a modulation parameterof the quantization step size, and the quantization step size decodingmeans uses the modulation parameter to modulate the quantization stepsize determined based on the parameter used by the prediction imagegeneration means in order to predict the quantization step size.

(Supplementary Note 28)

The video decoding device according to Supplementary note 27, whereinthe quantization step size decoding means adds a predetermined offset tothe quantization step size determined based on the parameter used by theprediction image generation means in order to predict the quantizationstep size.

(Supplementary Note 29)

A video encoding method for dividing input image data into blocks of apredetermined size, and applying quantization to each divided imageblock to execute a compressive encoding process, comprising: encoding aquantization step size that controls the granularity of quantization;using an image encoded in the past and a predetermined parameter togenerate a prediction image of an image block to be encoded; predictingthe quantization step size using the parameter used in generating theprediction image; and multiplexing an operational parameter used inencoding the quantization step size into the result of the compressiveencoding process.

(Supplementary Note 30)

The video encoding method according to Supplementary note 29, whereinthe operational parameter used in encoding the quantization step sizeincludes at least a flag representing whether to use the parameter upongeneration of the prediction image in order to control an operation forencoding the quantization step size based on the flag.

(Supplementary Note 31)

The video encoding method according to Supplementary note 29, whereinthe operational parameter used in encoding the quantization step sizecomprises at least a modulation parameter of the quantization step size,and upon encoding the quantization step size, the modulation parameteris used to modulate the quantization step size determined based on theparameter used in generating the prediction image in order to predictthe quantization step size.

(Supplementary Note 32)

The video encoding method according to Supplementary note 31, wherein apredetermined offset is added to the quantization step size determinedbased on the parameter used in generating the prediction image topredict the quantization step size.

(Supplementary Note 33)

A video decoding method for decoding image blocks using inversequantization of input compressed video data to execute a process ofgenerating image data as a set of the image blocks, comprising: decodinga quantization step size that controls the granularity of inversequantization; using an image decoded in the past and a predeterminedparameter to generate a prediction image of an image block to bedecoded; using a quantization step size assigned to a neighboring imageblock already decoded to predict the quantization step size upondecoding the quantization step size; de-multiplexing a bitstreamincluding an operational parameter used in decoding the quantizationstep size, and controlling an operation for decoding the quantizationstep size based on the de-multiplexed operational parameter.

(Supplementary Note 34)

The video decoding method according to Supplementary note 33, wherein atleast a flag representing whether to use the parameter used ingenerating the prediction image of the image block to be decoded isextracted as the operational parameter used in decoding the quantizationstep size, and the operation for decoding the quantization step size iscontrolled based on the flag.

(Supplementary Note 35)

The video decoding method according to Supplementary note 33, wherein atleast a modulation parameter of the quantization step size is extractedas the operational parameter used in decoding the quantization stepsize, and the modulation parameter is used to modulate the quantizationstep size determined based on the parameter used in generating theprediction image of the image block to be decoded in order to predictthe quantization step size.

(Supplementary Note 36)

The video decoding method according to Supplementary note 35, whereinupon decoding the quantization step size, a predetermined offset isadded to the quantization step size determined based on the parameterused in generating the prediction image of the image block to be decodedin order to predict the quantization step size.

(Supplementary Note 37)

A video encoding program for dividing input image data into blocks of apredetermined size, and applying quantization to each divided imageblock to execute a compressive encoding process, causing a computer toexecute: a process of encoding a quantization step size that controlsthe granularity of quantization; a process of using an image encoded inthe past and a predetermined parameter to generate a prediction image ofan image block to be encoded; a process of predicting the quantizationstep size using the parameter used in generating the prediction image;and multiplexing an operational parameter used in encoding thequantization step size into the result of the compressive encodingprocess.

(Supplementary Note 38)

The video encoding program according to Supplementary note 37, whereinthe operational parameter used in encoding the quantization step sizeincludes at least a flag representing whether to use the parameter upongeneration of the prediction image, and the computer is caused tocontrol an operation for encoding the quantization step size based onthe flag.

(Supplementary Note 39)

The video encoding program according to Supplementary note 37, whereinthe operational parameter used in encoding the quantization step sizeincludes at least a modulation parameter of the quantization step size,and upon encoding the quantization step size, the computer is caused touse the modulation parameter to modulate the quantization step sizedetermined based on the parameter used in generating the predictionimage in order to predict the quantization step size.

(Supplementary Note 40)

The video encoding program according to Supplementary note 39, whereinthe computer is caused to add a predetermined offset to the quantizationstep size determined based on the parameter used in generating theprediction image in order to predict the quantization step size.

(Supplementary Note 41)

A video decoding program for decoding image blocks using inversequantization of input compressed video data to execute a process ofgenerating image data as a set of the image blocks, causing a computerto execute: a process of decoding a quantization step size that controlsthe granularity of inverse quantization; a process of using an imagedecoded in the past and a predetermined parameter to generate aprediction image of an image block to be decoded; a process of using aquantization step size assigned to a neighboring image block alreadydecoded to predict the quantization step size upon decoding thequantization step size; a process of de-multiplexing a bitstreamincluding an operational parameter used in decoding the quantizationstep size, and a process of controlling an operation for decoding thequantization step size based on the de-multiplexed operationalparameter.

(Supplementary Note 42)

The video decoding program according to Supplementary note 41, causingthe computer to further execute: a process of extracting, as theoperational parameter used in decoding the quantization step size, atleast a flag representing whether to use the parameter used ingenerating the prediction image of the image block to be decoded; and aprocess of controlling an operation for decoding the quantization stepsize based on the flag.

(Supplementary Note 43)

The video decoding program according to Supplementary note 41, causingthe computer to further execute: a process of extracting, as theoperational parameter used in decoding the quantization step size, atleast a modulation parameter of the quantization step size; and aprocess of using the modulation parameter to modulate the quantizationstep size determined based on the parameter used in generating theprediction image of the image block to be decoded in order to predictthe quantization step size.

(Supplementary Note 44)

The video decoding program according to Supplementary note 43, whereinupon decoding the quantization step size, the computer is caused to adda predetermined offset to the quantization step size determined based onthe parameter used in generating the prediction image of the image blockto be decoded in order to predict the quantization step size.

(Supplementary Note 45)

A video encoding device for dividing input image data into blocks of apredetermined size, and applying quantization to each divided imageblock to execute a compressive encoding process, comprising quantizationstep size encoding means for encoding a quantization step size thatcontrols the granularity of quantization, wherein the quantization stepsize encoding means predicts the quantization step size that controlsthe granularity of quantization by using an average value ofquantization step sizes assigned to multiple neighboring image blocksalready encoded.

(Supplementary Note 46)

A video decoding device for decoding image blocks using inversequantization of input compressed video data to execute a process ofgenerating image data as a set of the image blocks, comprisingquantization step size decoding means for decoding a quantization stepsize that controls the granularity of inverse quantization, wherein thequantization step size decoding means predicts the quantization stepsize that controls the granularity of inverse quantization by using anaverage value of quantization step sizes assigned to multipleneighboring image blocks already encoded.

(Supplementary Note 47)

A video encoding method for dividing input image data into blocks of apredetermined size, and applying quantization to each divided imageblock to execute a compressive encoding process, comprising using anaverage value of quantization step sizes assigned to multipleneighboring image blocks already encoded to predict a quantization stepsize that controls the granularity of quantization.

(Supplementary Note 48)

A video decoding method for decoding image blocks using inversequantization of input compressed video data to execute a process ofgenerating image data as a set of the image blocks, comprising using anaverage value of quantization step sizes assigned to multipleneighboring image blocks already decoded to predict a quantization stepsize that controls the granularity of inverse quantization.

(Supplementary Note 49)

A video encoding program for dividing input image data into blocks of apredetermined size, and applying quantization to each divided imageblock to execute a compressive encoding process, causing a computer toexecute: a process of encoding a quantization step size that controlsthe granularity of quantization; and a process of using an average valueof quantization step sizes assigned to multiple neighboring image blocksalready encoded to predict the quantization step size that controls thegranularity of quantization.

(Supplementary Note 50)

A video decoding program for decoding image blocks using inversequantization of input compressed video data to execute a process ofgenerating image data as a set of the image blocks, causing a computerto execute: a process of decoding a quantization step size that controlsthe granularity of inverse quantization; and a process of using anaverage value of quantization step sizes assigned to multipleneighboring image blocks already decoded to predict the quantizationstep size that controls the granularity of inverse quantization.

While the present invention has been described with reference to theexemplary embodiments and examples, the present invention is not limitedto the aforementioned exemplary embodiments and examples. Variouschanges understandable to those skilled in the art within the scope ofthe present invention can be made to the structures and details of thepresent invention.

This application claims priority based on Japanese Patent ApplicationNo. 2011-51291, filed on Mar. 9, 2011, and Japanese Patent ApplicationNo. 2011-95395, filed on Apr. 21, 2011, the disclosures of which areincorporated herein in their entirety.

REFERENCE SIGNS LIST

-   -   10 quantization step size encoding unit    -   11 step size prediction unit    -   20 prediction image generation unit    -   30 predicted motion vector generation unit    -   50 quantization step size decoding unit    -   51 step size prediction unit    -   60 prediction image generation unit    -   70 predicted motion vector generation unit    -   101 frequency transformer    -   102 quantizer    -   103 variable-length encoder    -   104 quantization controller    -   105 inverse quantizer    -   106 inverse frequency transformer    -   107 frame memory    -   108 intra-frame predictor    -   109 inter-frame predictor    -   110 prediction selector    -   111 quantization step size prediction controller    -   112 multiplexer    -   201 variable-length decoder    -   202 inverse quantizer    -   203 inverse frequency transformer    -   204 frame memory    -   205 intra-frame predictor    -   206 inter-frame predictor    -   207 prediction selector    -   208 de-multiplexer    -   209 quantization step size prediction controller    -   1001 processor    -   1002 program memory    -   1003 storage medium    -   1004 storage medium    -   10311 quantization step size buffer    -   10312 entropy encoder    -   10313 predicted quantization step size generator    -   20111 entropy decoder    -   20112 quantization step size buffer    -   20113 predicted quantization step size generator

The invention claimed is:
 1. A video decoding device for decoding imageblocks based on inverse quantization of input compressed video data toexecute a process of generating image data as a set of the image blocks,comprising: a memory storing instructions; and at least one hardwareprocessor configured to execute the instructions to implement: decodinga quantization step size that controls a granularity of the inversequantization; generating a prediction image of an image block to bedecoded by using an image decoded in the past and a predeterminedparameter information on a prediction method of a prediction imagegeneration; and calculating the quantization step size that controls thegranularity of the inverse quantization by using a parameter used togenerate the prediction image and a mean value of at least a firstquantization step size assigned to a leftwardly adjacent neighboringimage block already decoded and a second quantization step size assignedto an upwardly adjacent neighboring image block already decoded.
 2. Avideo decoding method for decoding image blocks using inversequantization of input compressed video data to execute a process ofgenerating image data as a set of the image blocks, comprising: decodinga quantization step size that controls a granularity of the inversequantization; generating a prediction image of an image block to bedecoded by using an image decoded in the past and a predeterminedparameter information on a prediction method of a prediction imagegeneration; and calculating a quantization step size that controls agranularity of the inverse quantization by using a parameter used togenerate the prediction image and a mean value of at least a firstquantization step size assigned to a leftwardly adjacent neighboringimage block already decoded and a second quantization step size assignedto an upwardly adjacent neighboring image block already decoded.